Combining the functionality of multiple automatic cells by introducing a dual-arm robot station

The industrial revolution 4.0 is resulting in a shift of production and manufacturing processes to become more efficient and streamlined. This involves the refurnishing and upgrading of existing technology in production lines. Significant research done in the field of dual-arm robot manipulators shows us that anthropomorphic robots can be used interchangeably or replace humans, without major changes to the workplace, resulting in low cost, flexible automation and increased functionality. This thesis implementation aims to merge the functionalities of two automatic workstations by upgrading the elements of one of the workstations and removing the other workstation from the line. Based on the available resources and existent in-lab facilities, the prime objective is to determine the most efficient means to utilize a dual-arm manipulator to combine and update the functionalities of these two individual cells. And, how best to create exposure, by demonstrating the operations and features of the relatively unacquainted breed of industrial robots to new generations of students. This thesis focuses on the virtual implementation of the dual-arm station and the process flow is modelled and simulated, as proof of concept. The physical implementation would be carried out subsequently. Based on an extensive analysis of relevant literature, manipulator capabilities, in-house fabrication and purchasing feasibilities; the most efficient implementation was made after multiple iterative changes in cell and tooling design, and the process was created using the offline programming tool. The created dual-arm cell has replaced two individual robot manipulators and additional conveyor systems and acts as both, a production and an Automatic Storage and Retrieval System (AS/RS) unit, additionally it enables the intelligent production line (where the cell is being installed) to interact with other isolated intelligent systems in the laboratory. The introduced changes to the production line have increased its functionality, while removing obsolete equipment. The production efficiency is increased while reducing the space utilized and demonstrates the operation of a dual-arm industrial robots. Future works to improve the operations in the cell may include using anthropomorphic hands as tooling and utilizing intelligent vision systems to additionally handle deformable linear objects for the REMODEL project of the Horizon 2020 research and innovation programming, while still serving its originally intended purpose in the intelligent production line

An approach for the bimanual manipulation of a deformable linear object using a dual-arm industrial robot : cable routing use case

The automation of processes that handle deformable materials is considered to be a complicated task. Due to their properties, these materials require specialised solutions for their manipulation, using robotic systems and mostly, using specifically developed hardware which limits its use for different deformable objects. To solve this issue, this paper presents an approach for bimanually manipulating Deformable Linear Objects (DLOs) using a dual-arm industrial robot. This approach aims at providing an automatic, generic, and easily reconfigurable solution and is implemented for routing cables in a human-centric platform. The approach consists of a cyber-physical system (CPS) composed by commercial hardware: a robot equipped with two parallel grippers, and a reconfigurable Robot Operating System (ROS) software. In more details, the developed software extracts information about the process, such as the routing path, keypoints of the workstation setup and objects dimensions. Then, it uses the extracted information to generate suitable bimanual trajectories for the robot. Finally, the approach has been tested for three different routine paths.acceptedVersionPeer reviewe

A method for understanding and digitizing manipulation activities using programming by demonstration in robotic applications

Robots are flexible machines, where the flexibility is achieved, mainly, by the re-programming of the robotic system. To fully exploit the potential of robotic systems, an easy, fast, and intuitive programming methodology is desired. By applying such methodology, robots will be open to a wider audience of potential users (i.e. SMEs, etc.) since the need for a robotic expert in charge of programming the robot will not be needed anymore. This paper presents a Programming by Demonstration approach dealing with high-level tasks taking advantage of the ROS standard. The system identifies the different processes associated to a single-arm human manipulation activity and generates an action plan for future interpretation by the robot. The system is composed of five modules, all of them containerized and interconnected by ROS. Three of these modules are in charge of processing the manipulation data gathered by the sensors system, and converting it from the lowest level to the highest manipulation processes. In order to do this transformation, a module is used to train the system. This module generates, for each operation, an Optimized Multiorder Multivariate Markov Model, that later will be used for the operations recognition and process segmentation. Finally, the fifth module is used to interface and calibrate the system. The system was implemented and tested using a dataglove and a hand position tracker to capture the operator’s data during the manipulation. Four users and five different object types were used to train and test the system both for operations recognition and process segmentation and classification, including also the detection of the locations where the operations are performed.Peer reviewe

Extending the motion planning framework—MoveIt with advanced manipulation functions for industrial applications

MoveIt is the primary software library for motion planning and mobile manipulation in ROS, and it incorporates the latest advances in motion planning, control and perception. However, it is still quite recent, and some important functions to build more advanced manipulation applications, required to robotize many manufacturing processes, have not been developed yet. MoveIt is an open source software, and it relies on the contributions from its community to keep improving and adding new features. Therefore, in this paper, its current state is analyzed to find out which are its main necessities and provide a solution to them. In particular, three gaps of MoveIt are addressed: the automatic tool changing at runtime, the generation of trajectories with full control over the end effector path and speed, and the generation of dual-arm trajectories using different synchronization policies. These functions have been tested with a Motoman SDA10F dual-arm robot, demonstrating their validity in different scenarios. All the developed solutions are generic and robot-agnostic, and they are openly available to be used to extend the capabilities of MoveIt.publishedVersionPeer reviewe

Combining the functionality of multiple automatic cells by introducing a dual-arm robot station

The industrial revolution 4.0 is resulting in a shift of production and manufacturing processes to become more efficient and streamlined. This involves the refurnishing and upgrading of existing technology in production lines. Significant research done in the field of dual-arm robot manipulators shows us that anthropomorphic robots can be used interchangeably or replace humans, without major changes to the workplace, resulting in low cost, flexible automation and increased functionality. This thesis implementation aims to merge the functionalities of two automatic workstations by upgrading the elements of one of the workstations and removing the other workstation from the line. Based on the available resources and existent in-lab facilities, the prime objective is to determine the most efficient means to utilize a dual-arm manipulator to combine and update the functionalities of these two individual cells. And, how best to create exposure, by demonstrating the operations and features of the relatively unacquainted breed of industrial robots to new generations of students. This thesis focuses on the virtual implementation of the dual-arm station and the process flow is modelled and simulated, as proof of concept. The physical implementation would be carried out subsequently. Based on an extensive analysis of relevant literature, manipulator capabilities, in-house fabrication and purchasing feasibilities; the most efficient implementation was made after multiple iterative changes in cell and tooling design, and the process was created using the offline programming tool. The created dual-arm cell has replaced two individual robot manipulators and additional conveyor systems and acts as both, a production and an Automatic Storage and Retrieval System (AS/RS) unit, additionally it enables the intelligent production line (where the cell is being installed) to interact with other isolated intelligent systems in the laboratory. The introduced changes to the production line have increased its functionality, while removing obsolete equipment. The production efficiency is increased while reducing the space utilized and demonstrates the operation of a dual-arm industrial robots. Future works to improve the operations in the cell may include using anthropomorphic hands as tooling and utilizing intelligent vision systems to additionally handle deformable linear objects for the REMODEL project of the Horizon 2020 research and innovation programming, while still serving its originally intended purpose in the intelligent production line

An Approach for Modeling Grasping Configuration Using Ontology-based Taxonomy

The handling of flexible materials is complicated task in robotics and automation. Due to deformability and fragility of flexible materials, robots are equipped with the state-of-the-art sensors and grippers to perform such tasks. Nonetheless, industry still lacks for approaches and techniques for handling these materials. Therefore, several industries and mass production systems require hiring human to perform the deformable materials-related task. These tasks might include usage of toxic martials (e.g. carbon fiber sheets) or dangerous tools (e.g. sharp cutting knives). In this regard, this paper presents an approach for selecting grasping configuration of objects based on the product’s properties such as rigidity, surface roughness and shape, and the required task. Briefly, this research is based on several published taxonomies for modeling the hand of the human while grasping different objects. After refining the taxonomy, an ontology model is populated which will be queried for specifying the gripper’s properties such as number of fingers and required grasping force that can perform the selected task on the selected product. Finally, this research presented a use case form the REMODEL (Robotic tEchnologies for the Manipulation of cOmplex DeformablE Linear objects) project in order to assess and validate the approach. For the future, this research expected to include the selection of the grippers, the robotic arm and its approach for grasping the product.acceptedVersionPeer reviewe

A Machine-Learning-Based Analysis of the Relationships between Loneliness Metrics and Mobility Patterns for Elderly

Loneliness and social isolation are subjective measures associated with the feeling of discomfort and distress. Various factors associated with the feeling of loneliness or social isolation are: the built environment, long-term illnesses, the presence of disabilities or health problems, etc. One of the most important aspect which could impact feelings of loneliness is mobility. In this paper, we present a machine-learning based approach to classify the user loneliness levels using their indoor and outdoor mobility patterns. User mobility data has been collected based on indoor and outdoor sensors carried on by volunteers frequenting an elderly nursing house in Tampere region, Finland. The data was collected using Pozyx sensor for indoor data and Pico minifinder sensor for outdoor data. Mobility patterns such as the distance traveled indoors and outdoors, indoor and outdoor estimated speed, and frequently visited clusters were the most relevant features for classifying the user’s perceived loneliness levels.Three types of data used for classification task were indoor data, outdoor data and combined indoor-outdoor data. Indoor data consisted of indoor mobility data and statistical features from accelerometer data, outdoor data consisted of outdoor mobility data and other parameters such as speed recorded from sensors and course of a person whereas combined indoor-outdoor data had common mobility features from both indoor and outdoor data. We found that the machine-learning model based on XGBoost algorithm achieved the highest performance with accuracy between 90% and 98% for indoor, outdoor, and combined indoor-outdoor data. We also found that Lubben-scale based labelling of perceived loneliness works better for both indoor and outdoor data, whereas UCLA scale-based labelling works better with combined indoor-outdoor data.publishedVersionPeer reviewe

An approach based on machine vision for the identification and shape estimation of deformable linear objects

The automation of processes that handle deformable materials, and in particular Deformable Linear Objects (DLOs), such as cables, ropes, and sutures; is a challenging task. Due to their properties, it is very difficult to predict the shape of these objects, making indispensable the use of perception systems for their manipulation. However, the detection of a DLO is a non-trivial task, and it can be even more complicated when additional considerations are made, such as detecting multiple DLOs, with small distances between them or even adjacent to each other, and with occlusions and entanglements between them. In this paper, a novel machine vision approach for estimating the shape of DLOs is proposed to address all these challenges. This approach processes the different DLOs in the image sequentially, repeating the following procedure for each of them. First, the DLO is segmented by examining the colors and edges in the image. Next, the remaining pixels are analyzed using evaluation windows to identify a series of points along the DLO’s skeleton. These points are then employed to model the DLO’s shape using a polynomial function. Finally, the output is evaluated by an unsupervised self-critique module, which validates the results, or fine-tunes the system’s parameters and repeats the process. The performance of the system was tested with several wiring harnesses, detecting all their cables in homogeneous and complex backgrounds, with adjacent cables, and with occlusions. The results show an outstanding performance, with a successful shape estimation rate of more than 90% for some of the system configurations.Peer reviewe

Deformable Objects Grasping and Shape Detection with Tactile Fingers and Industrial Grippers

In this paper, a cyber-physical system composed by a tactile sensor, a robotic gripper and suitable ROS software nodes is proposed. The tactile sensors are shown to be compatible with three different commercial grippers, and the developed ROS nodes for the data acquisition and elaboration enable the implementation of complex tasks such as the grasping and the shape reconstruction of deformable linear objects like cables. The effectiveness of the systems is tested with cable of different diameters and with wiring harnesses composed by several cables grouped together, focusing on the reconstruction of linear and quadratic curves representing the cable shape. Experimental trials are also executed to show the possibility of exploiting the shape reconstruction provided by the proposed system to correct the gripper grasping pose.acceptedVersionPeer reviewe

Managing Perceived Loneliness and Social-Isolation Levels for Older Adults: A Survey with Focus on Wearables-Based Solutions

As an inevitable process, the number of older adults is increasing in many countries worldwide. Two of the main problems that society is being confronted with more and more, in this respect, are the inter-related aspects of feelings of loneliness and social isolation among older adults. In particular, the ongoing COVID-19 crisis and its associated restrictions have exacerbated the loneliness and social-isolation problems. This paper is first and foremost a comprehensive survey of loneliness monitoring and management solutions, from the multidisciplinary perspective of technology, gerontology, socio-psychology, and urban built environment. In addition, our paper also investigates machine learning-based technological solutions with wearable-sensor data, suitable to measure, monitor, manage, and/or diminish the levels of loneliness and social isolation, when one also considers the constraints and characteristics coming from social science, gerontology, and architecture/urban built environments points of view. Compared to the existing state of the art, our work is unique from the cross-disciplinary point of view, because our authors’ team combines the expertise from four distinct domains, i.e., gerontology, social psychology, architecture, and wireless technology in addressing the two inter-related problems of loneliness and social isolation in older adults. This work combines a cross-disciplinary survey of the literature in the four aforementioned domains with a proposed wearable-based technological solution, introduced first as a generic framework and, then, exemplified through a simple proof of concept with dummy data. As the main findings, we provide a comprehensive view on challenges and solutions in utilizing various technologies, particularly those carried by users, also known as wearables, to measure, manage, and/or diminish the social isolation and the perceived loneliness among older adults. In addition, we also summarize the identified solutions which can be used for measuring and monitoring various loneliness- and social isolation-related metrics, and we present and validate, through a simple proof-of-concept mechanism, an approach based on machine learning for predicting and estimating loneliness levels. Open research issues in this field are also discussed.publishedVersionPeer reviewe